How to design a powerful family of Particle Swarm Optimizers for inverse modeling

In this paper we show how to design a powerful set of Particle Swarm optimizers to be applied in inverse modeling. The design is based on the interpretation of the swarm dynamics as a stochastic damped mass-spring system, the so-called PSO continuous model. Based on this idea we derived a family of PSO optimizers (GPSO, CC-PSO and CP-PSO) having different exploitation and exploration capabilities. Their convergence is related to the stability of their first (mean trajectories) and second order moments (variance and temporal covariance). Good parameter sets are located inside their first stability regions close to the upper border of their respective second stability regions where the attraction from the particles oscillation center is very weak. In this region of weak attraction, both convergence to the global minimum and exploration of the search space are possible. Based on this idea we have designed a particle-cloud algorithm where each particle in the swarm has different inertia (damping) and acceleration (rigidity) constants. We explored the performance of these algorithms for different PSO members using different benchmark functions, showing that the cloud algorithms have a very good balance between exploration and exploitation. Also, the cloud design helps to avoid two main drawbacks of the PSO algorithm: the tuning of the PSO parameters and the clamping of the particles velocities. We also present the lime-and-sand algorithm that changes the time step with iterations. This feature helps to avoid entrapment in local minima when the time step is increased, and enables exploring around the global best when the time step is decreased. All these designs are based on the theoretical analysis of the PSO dynamics. We explain how to use this knowledge to the solution and appraisal of inverse problems. Finally we briefly introduce the combined used of PSO and model reduction techniques to allow posterior sampling in high dimensional spaces.